Search for: All records

A high-quality hydrographic observational database is essential for ocean and climate studies and operational applications. Because there are numerous global and regional ocean databases, duplicate data continues to be an issue in data management, data processing and database merging, posing a challenge on effectively and accurately using oceanographic data to derive robust statistics and reliable data products. This study aims to provide algorithms to identify the duplicates and assign labels to them. We propose first a set of criteria to define the duplicate data; and second, an open-source and semi-automatic system to detect duplicate data and erroneous metadata. This system includes several algorithms for automatic checks using statistical methods (such as Principal Component Analysis and entropy weighting) and an additional expert (manual) check. The robustness of the system is then evaluated with a subset of the World Ocean Database (WOD18) with over 600,000in-situtemperature and salinity profiles. This system is an open-source Python package (named DC_OCEAN) allowing users to effectively use the software. Users can customize their settings. The application result from the WOD18 subset also forms a benchmark dataset, which is available to support future studies on duplicate checks, metadata error identification, and machine learning applications. This duplicate checking system will be incorporated into the International Quality-controlled Ocean Database (IQuOD) data quality control system to guarantee the uniqueness of ocean observation data in this product. 

Ocean temperature observations are crucial for a host of climate research and forecasting activities, such as climate monitoring, ocean reanalysis and state estimation, seasonal-to-decadal forecasts, and ocean forecasting. For all of these applications, it is crucial to understand the uncertainty attached to each of the observations, accounting for changes in instrument technology and observing practices over time. Here, we describe the rationale behind the uncertainty specification provided for all in situ ocean temperature observations in the International Quality-controlled Ocean Database (IQuOD) v0.1, a value-added data product served alongside the World Ocean Database (WOD). We collected information from manufacturer specifications and other publications, providing the end user with uncertainty estimates based mainly on instrument type, along with extant auxiliary information such as calibration and collection method. The provision of a consistent set of observation uncertainties will provide a more complete understanding of historical ocean observations used to examine the changing environment. Moving forward, IQuOD will continue to work with the ocean observation, data assimilation and ocean climate communities to further refine uncertainty quantification. We encourage submissions of metadata and information about historical practices to the IQuOD project and WOD.